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1 3  ABSTRACT  (Maximum  200  words) 


This  paper  discusses  progress  at  the  Hawaii  Laboratory  over  the  past  three  years  in  the  development  of  three 
teleoporated  systems:  the  Advanced  Tethered  Vehicle  (ATV),  the  TeleOperated  Vehicle  (TOV)  and  the  TeleOperated/ 
telepresence  System  (TOPS).  We  will  comment  on  experience  to  date  with  these  systems,  pointing  to  the  need  for  additional 
sensory  information  by  the  remote  system  operator  to  achieve  increased  performance.  Each  of  these  systems  addresses  a  dif¬ 
ferent  mission,  but  all  share  common  approaches  to  subsystem  design  and  to  various  degrees  incorporate  remote  presence. 
ATV  and  TOV  utilize  a  fiber  optic  data  link  for  high  band  width,  small  size,  ruggedness,  and  in  the  case  of  the  TOV,  jam 
resistance  and  covertness. 
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ADVANCES  AND  EXPERIENCE  WITH  TELEOPERATED  SYSTEMS  INCORPORATING 

REMOTE  PRESENCE 

D.  W.  Murphy 

Naval  Ocean  Systems  Center,  Hawaii  Laboratory 


INTRODUCTION 

The  Naval  Ocean  Systems  Center 
(NOSC)  Hawaii  Laboratory  has  a  long 
standing  program  in  research  and 
development  on  teleoperated  systems. 
Recent  work  is  summarized  in  the  literature 
[1,  2,  3].  Our  interest  in  teleoperated 
systems  is  removing  the  human  from 
hazardous  environments.  Our  research  and 
development  in  this  area  has  the  objective  of 
maximizing  system  capabilities  and 
minimizing  operator  work  load  and  training 
by  providing  the  system  operator  with  a 
sense  of  remote  presence.  Remote  presence 
can  be  described  as: 

"..the  perception  of  actually  being  t  the 
remote  location." 

The  degree  of  'presence'  achieved  is 
determined  by:  (1)  the  fidelity  and/or 
richness  of  the  sensory  information,  (2)  the 
ability  to  control  the  relation  of  the  sensors 
to  the  remote  environment,  and  (3)  the 
ability  to  physically  interact  with  the  remote 
environment. 

The  cost  of  teleoperation  in  a  system 
is  increased  complexity  and,  based  on 
present  technology,  a  reduced  capability  for 
action.  Complexity  arises  from  the 
requirement  to  provide  the  remote  operator 
with  sensory  and  control  interfaces. 
Reduced  capability  for  action  is  a  result  of 
the  reduced  information  to  the  operator 
resulting  from  the  limitations  of  the  sensors, 
displays,  and  control  mechanisms.  One 
focus  of  our  research  is  to  improve  these 
technologies. 

This  paper  discusses  progress  at  the 
Hawaii  Laboratory  over  the  past  three  years 
in  the  development  of  three  teleoperated 
systems:  the  Advanced  Tethered  Vehicle 
(ATV),  the  TeleOperated  Vehicle  (TOV1  and 
the  TeleOperator/telePresence  System 
(TOPS).  We  vul  comment  on  experience  to 


date  with  these  systems,  pointing  to  the  need 
for  additional  sensory  information  by  the 
remote  system  operator  to  achieve  increased 
performance.  Each  of  these  systems 
addresses  a  different  mission,  but  all  share 
common  approaches  to  subsystem  design 
and  to  various  degrees  incorporate  remote 
presence.  ATV  and  TOV  utilize  a  fiber  optic 
data  link  for  high  band  width,  small  size, 
ruggedness,  and  in  the  case  of  the  TOV,  jam 
resistance  and  covertness. 

DISCUSSION 

Advanced  Tethered  Vehicle 
(ATV) 

The  objective  of  the  ATV  is  to 
provide  the  Navy  with  the  capability  to 
perform  work  to  ocean  depths  of  20,000 
feet.  ATV  is  a  self-contained,  remotely 
operated  undersea  system  [4].  Since  ATV 
will  be  operated  and  maintained  by  Navy 
personnel  the  design  was  based  on  mature 
technology  to  insure  a  high  level  of  system 
reliability  and  availability.  The  ATV  system, 
figure  1,  consists  of  five  main  components: 
a  neutrally  buoyant  submersible  vehicle, 
tether  cable,  handling  system,  control  station 
and  auxiliary  equipment 

The  vehicle  work  package,  figure  2, 
consists  of  two  manipulators,  four  television 
(TV)  cameras,  a  35-mm  film  camera  and 
strobe  light,  and  a  variety  of  tools  and 
lights.  The  manipulators  are  identical 
master-slave  units  incorporating  force 
feedback.  Of  the  four  TV  cameras,  two  are 
used  in  a  stereo  pair,  one  has  a  zoom  lens, 
and  one  acts  as  a  view  finder  for  the  film 
camera.  The  stereo  pair  and  zoom  cameras 
are  mounted  on  pan  and  tilt  units.  A  suite  of 
hydraulically  powered  tools  is  mounted  on 
the  support  tray  below  the  manipulators  and 
includes  a  rotai7  drive  to  which  an  abrasive 
cutoff  wheel,  drill,  or  impact  wrench  can  be 
attached  and  independent  linear  tools 
consisting  ot  a  spreader,  cable  cutter,  jack 
and  grabber. 
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Fig.  1  A  TV  System  Configuration 

re  »  «■  -•«.  .a  r  aavi 


Fig.  3  I'OV  and  Control  Van 


The  subsystems  supporting 
teleoperation,  i.e.  sensors,  displays, 
telemetry,  and  control  interfaces  are 
summarized  in  Table  1.  The  primary  control 
interface  with  the  vehicle  are  through  its  joy¬ 
stick  propulsion  control  and  the  dual,  force 
feedback  manipulator  controllers.  The 
primary  sensory  displays  include  the  flat 
panel  stereo  TV  display,  the  sonar  display  , 
the  navigation  display  and  force  feedback 
from  the  manipulator  controllers.  Over  240 
hours  of  operation  at  depths  to  20,600  ft 
have  been  logged  on  the  system  during  test 
and  evaluation.  Experience  to  date  has 
demonstrated  that  the  system  meets  its 
reliability  and  effectiveness  objectives. 
Operations  with  the  work  system  have 
demonstrated  that  the  operators  can 
effectively  perform  the  types  of  tasks 
required  for  deep  ocean  work,  e.g. 
attachment,  object  manipulation,  tool 
operation.etc. 

TeleOperated  Vehicle 
(TOY) 

The  objective  of  the  TOV  is  to 
provide  Marine  Corps  and  Army  personnel 
with  a  prototype  unmanned  ground  vehicle 
(UGV)  system  which  can  be  used  in  training 
exercises  and  operations  to  assess  UGV 
systems’  military  worth,  develop  concepts 
of  employment,  and  develop  an  experience 
base  to  guide  future  development. 

The  TOV  is  a  remotely  operated 
system  which  permits  an  operator  to  extend 
his  sensory,  motor  function,  and  problem¬ 
solving  skills  to  a  vehicle  located  up  to  30 
kilometers  away  |5.  6|.  The  TOV  system 
consists  of  a  Remote  Vehicle  and  a  Control 
Station,  figure  3.  An  operator  controls  the 
TOV  system  using  displays  and  controls 
located  in  the  Control  Station.  Visual  and 
auditory  displays  have  been  designed  to 
facilitate  understanding  ot  the  vehicles 
situation.  Hand  and  foot  operated  controls, 
analogous  to  automotive  vehicle  controls, 
are  used  to  control  vehicle  functions.  A  fiber 
optic  data  link  allows  physical  separation  of 
the  operator  from  hazards  encountered  by 
the  remote  vehicle.  Based  on  the  High 
Mobiiitv  Multi  Wheeled  Venicie 
iHMMWV).  TOV  is  capable  or  high  transit 
speeds  and  can  traverse  severe  off  road 
terrain.  A  mission-specific,  add-on 


subsystem  for  Reconnaissance, 
Surveillance,  and  Target  Acquisition 
(RSTA)  has  been  developed. 

The  subsystems  supporting 
teleoperation  of  TOV  are  summarized  in 
Table  1.  The  control  station  specified  in  the 
table  provides  an  interface  similar  to  what  is 
provided  in  the  actual  vehicle,  i.e.  steering 
wheel,  throttle,  gear  shift,  and  seat.  These 
controls  are  presented  to  the  operator  in  the 
same  relative  position  they  occupy  in  the 
HMMWV,  figure  4.  An  alternate  controller 
has  been  developed  which  fits  in  several 
man  portable  containers  and  is  easily  set  up 
and  operated  in  the  field,  figure  5.  Driving 
controls  are  presented  to  the  operator  using  a 
motorcycle  type  controller  that  integrates  the 
throttle,  brakes,  gear  shift,  ignition  and 
starting  controls.  This  design  was  selected 
after  extensive  field  testing  which 
considered  joy-stick  steering,  steering 
wheel,  and  the  motorcycle  handle  bar  like 
steering.  The  integrated  motorcycle-like 
system  was  selected  based  on  performance, 
compactness  and  ease  of  use  in  the  field. 
Both  control  approaches  use  a  helmet 
mounted,  stereo  display.  This  display  is 
described  extensively  in  references  [7,  8. 9]. 

NOSC  has  constructed  three  TOV 
systems.  Two  systems  are  operated  in 
support  of  military  exercises  and 
demonstrations  while  the  third  remains  at  the 
NOSC  Hawaii  Laboratory  where  it  supports 
further  research  and  development.  As  a 
result  of  operation  of  these  systems  an 
extensive  experience  base  has  been 
developed. 

Vision  is  the  primary  sensory  mode 
for  remote  driving.  Extensive  research  has 
been  performed  to  evaluate  various  types  of 
displays  and  to  determine  their  influence  on 
vehicle  operation  [8,  9].  Field  testing  with 
the  vehicles  has  provided  additional  insight 
into  vision  system  performance  and 
requirements.  For  effective  high  speed 
driving  in  rough  terrain,  head  coupled  stereo 
vision  is  important  to  performance  for 
recognition  of  terrain  features,  such  a.-> 
rocks,  depressions  and  ditches  [8,  9|.  Field 
of  view  (FOV)  is  also  important  since  we 
obtain  most  of  our  navigation  cues  from 
peripheral  vision.  For  existing  sensors  and 


Fig.  4  TOV  Control  Station 


Fig,  5  TOV  Portable  Control  Station 


displays,  FOV  is  a  trade-off  for  display 
resolution.  Experience  at  NOSC  indicates 
that  if  offered  a  40°  FOV  with  relatively 
high  resolution  or  a  55°  FOV  with  lower 
resolution  a  vehicle  operator  will  select  the 
higher  resolution  display  if  head  coupling  is 
incorporated.  Our  experience  and  that  of 
others  [10}  indicates  that  fora  fixed  viewing 
system,  a  40°  FOV  is  inadequate  for  driving 
in  complex  environments.  With  a  head- 
coupled  system  operators  typically  pan  the 
viewing  system  to  get  a  higher  effective 
FOV.  Head  coupling  appears  to  provide  ine 
required  spatial  awareness.  Color  vision  is 
also  preferred  over  monochrome  when 
driving  in  rough  terrain  [8.  10|  because  it 


provides  increased  information  to  the 
operator  allowing  discrimination  of  objects 
in  grass,  depressions,  shadows,  etc.  It  is 
important  that  sensory  displays  be  in  one  to 
one  correspondence  with  what  is  sensed  on 
the  vehicle.  This  is  especially  true  for  the 
display  system  magnification  and  head 
motion. 

Experience  gained  with  these 
systems  indicates  that  for  performance  and 
safety  the  operators  require  additional 
sensory  data  on  the  remote  vehicles'  roll  and 
pitch  orientation  (situational  awareness).  An 
intuitive  display  of  vehicle  attitude  is 
required  to  allow  the  operator  to  perceive 
vehicle  attitude  on  hills  and  side  slopes. 
TOV  operators  have  experienced  several 
near  accidents  because  they  were  not  aware 
of  the  vehicle  being  on  a  steep  grade. 
Experience  at  Sandia  National  Laboratory 
also  indicates  that  the  majority  of  accidents 
with  teleoperated  vehicles  can  be  attributed 
to  a  lack  of  situational  awareness  [10]. 

TeleOperator/telePresence  Svstem 
(TOPS) 

The  objective  of  the  TOPS  project  is 
to  develop  an  anthropomorphic  manipulator 
system  and  a  high  resolution  stereo  vision 
system  to  provide  a  remote  work  capability 
equivalent  to  an  undersea  diver.  The 
manipulator  system  has  been  developed  by  a 
team  made  up  of  SARCOS  Research  Corp. 
and  the  Center  for  Engineering  Design  at  the 
University  of  Utah.  The  helmet-mounted 
display  system  was  produced  by  Wright- 
Patterson  Air  Force  Base  based  on  their 
work  in  the  LHX  Helicopter  program. 
Figure  6  shows  the  master  and  slave  system 
with  the  operator  using  the  exoskeletal 
controller  and  the  helmet  mounted  display. 
The  manipulator  system  includes  a  16 
degree  of  freedom  (DOF)  hand/arm.  a  three 
DOF  torso  and  a  three  DOF  neck  to  provide 
the  telerobot  with  the  job  site  mobility  to 
perform  effective  work. 

Teleoperation  subsystems  are 
summarized  in  Table  1.  The  remote 
manipulator  (slave)  is  kinematically 
equivalent  to  a  human.  Tbp  manipulator  and 
exoskeletal  controller  incorporate  bilateral 
force  and  position  feedback,  i.e.  the  force 
and  position  of  one  are  reflected  at  the  other. 


Force  and  position  sensors  are  mounted  at 
all  the  joints. 


Fig.  6  TOPS  manipulator  and  controller 


The  manipulator  system  is  presently 
in  the  refinement  and  tuning  stage  after 
which  a  full  range  of  laboratory  testing  is 
planned.  Anecdotal  information  from  initial 
testing  of  the  display  system  with  a  seven 
DOF  manipulator  available  at  the  Hawaii 
Laboratory  indicates  that  it  provides  the 
operator  with  a  sense  of  remote  presence. 
For  these  tests,  the  stereo  camera  pair  was 
mounted  on  a  simple  pan  and  tilt  unit  that 
followed  the  operator’s  head  motions. 
Insights  gained  to  date  include:  (1)  the 
operators  would  prefer  a  manipulator  with 
an  additional  degree  of  freedom  at  the 
shoulder  (abduction/adduction),  (2)  the 
operators  desired  the  ability  to  move  the 
camera  laterally  and  fore  and  aft  (torso 
motion)  to  get  differing  perspectives  on  the 
task,  and  (3)  the  tests  demonstrated  that  the 
high  resolution  stereo  display  significantly 
enhanced  the  ability  to  perform  some  tasks. 

SUMMARY  AND  FUTURE  DIRECTION 

The  level  of  presence  required  in  a 
teleoperated  system  is  a  function  of  its 
intended  use.  It  has  been  shown  that 
through  inclusion  of  remote  presence 
technology  the  task  of  operating  a  telerobot 
can  be  simplified  and  made  more  efficient. 
We  have  found  that  remote  ground  vehicle 
operators  require  additional  sensory 
information,  particularly  in  the  area  of 
attitude  and  location.  Operators  easily  get 
lost,  and  have  no  feeling  for  vehicle  attitude 
or  location. 


There  is  a  need  for  improved  vision 
technology.  Remote  work  requires  good 
resolution  and  can  accept  a  limited  field  of 
view.  High  speed  driving  of  remote  vehicles 
requires  high  central  resolution  and  a  wide 
FOV.  TOV  system  drivers  have  the 
equivalent  of  20/95  vision.  Visual  acuity  of 
20/40  is  the  minimum  required  by  many 
U.S.  driving  laws  [8].  Small,  high 
resolution,  low  cost,  color  displays  are 
required. 

NOSC  will  continue  to  develop 
teleoperators  for  the  U.S.  Department  of 
Defense.  Our  experience  has  shown  that  the 
design  of  the  operator  interface  is  critical  to 
the  success  and  performance  of  these 
systems.  One  approach  we  wish  to 
investigate  is  the  use  of  'Virtual  World' 
technology  to  fuse  sensory  data  and  present 


it  to  the  operator  in  an  integrated  context  that 
is  natural  for  the  task.  The  first  application 
under  consideration  for  this  approach  is  in 
integrating  the  information  available  to 
undersea  teleoperated  vehicle  operators  from 
their  sonar,  navigation  and  video  sensors. 
Conceptually  this  information  would  be 
presented  to  the  operator  through  a  head 
mounted  display  system,  such  as  is  used  in 
TOPS,  but  the  scene  would  be  computer 
generated.  The  scene  presented  to  the  wearer 
would  be  a  graphical  representation  of  the 
world  model  derived  from  the  combined 
sensor  inputs.  Since  the  operator  would  be 
looking  into  the  model  from  a  computer 
generated  vantage  point  he  could  choose  his 
point  of  view  to  be  on  the  vehicle  or  outside 
of  it  to  navigate  and  maintain  an  intuitive 
sense  of  where  the  vehicle  is  and  has  been. 
This  ability  to  select  a  view  point  should  aid 
in  mission  planning  and  route  planning. 


TABLE  1 

SUBSYSTEM  SPECIFICATIONS 


SUBSYSTEM 

AT  V 

TOV 

TOPS 

MOBILITY 

CONTROL 

Joy  Stick  -  4  DOF 

HMMWV  Replica 

Exoskeletal 

NAVIGATION 

Long  Base  Line 

SatNav 

None 

Display 

Graphic  -  Plan  view 

Graphic 

SENSORY 

DISPLAY 

VISION 

Monochrome,  512  lines 

Monochrome,  512  lines 

Monochrome.  1023  In. 

Display 

Flat  Panel  Stereo 

Helmet  Mounted  Stereo 

Helmet  Mounted  Stereo 

FOV 

60  Degrees 

40  Degrees 

70  Degrees 

Direction 

Joy  Stick  Directed 

Helmet  Tracking 

Helmet  Tracking 

AUDIO 

Monaural 

Binaural 

Binaural 

Manipulation 

Terminus  Control 

Force  feedback 

5  lb  resolution 

None 

Exoskeletal  Con'rol 

Force  &  Position 

1 .5  oz  resolution 

MANIPULATION 

Manufacturer 

Western  Space  &  Marine 

NONE 

SARCOS 

DEGREES  OF  FREEDOM 

7  Total 

22  Total 

Neck 

- 

3 

Torso 

- 

3 

Arm 

6 

7 

Hand 

1 

9 

DATA  LINK 

TYPE 

Fiber  Optic 

Fiber  Optic 

.Analog  Cable 

Data  Rate 

200  Mbs  Bidirectional 

200  Mbs  Bidirectional 

800  Mbs  Equivalent 

LENGTH 

23000  ft 

30  Km 

50  ft 
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